Effects of Data-Based Decision-Making on Low-Performing Readers in General Education Classrooms: Cumulative Evidence From Six Intervention Studies.

In most general education classrooms in Germany, students with and without special educational needs are taught together. To support teachers in adapting instruction to these heterogeneous classrooms, we have developed learning progress assessment (LPA) and reading instructional materials, the Reading Sportsman (RS), in line with the theoretical framework of data-based decision-making, which has led to beneficial effects in several studies. However, data from these studies have not been analyzed to examine effects for low-performing readers. Data within and across six quasi-experimental studies conducted by our team in Grades 2 to 4 were reanalyzed to examine the effects of LPA on students whose reading fluency scores were at or below the 25th percentile (n = 1,346 students from 264 classes). In each study, students had been assigned to a control group (with business-as-usual instruction), an LPA group only, or an LPA-RS group (i.e., with teachers provided with LPA and the RS). Separate multilevel regression results were integrated by means of meta-analytical methods to investigate the consistency of results. Overall, findings from the single studies indicated no positive effects of LPA with or without the RS compared with the control group. The integrated analyses showed small positive effect trends on reading fluency and intrinsic reading motivation.

How Much g Is in the Distractor? Re-Thinking Item-Analysis of Multiple-Choice Items.

Distractors might display discriminatory power with respect to the construct of interest (e.g., intelligence), which was shown in recent applications of nested logit models to the short-form of Raven's progressive matrices and other reasoning tests. In this vein, a simulation study was carried out to examine two effect size measures (i.e., a variant of Cohen's ω and the canonical correlation RCC) for their potential to detect distractors with ability-related discriminatory power. The simulation design was adopted to item selection scenarios relying on rather small sample sizes (e.g., N = 100 or N = 200). Both suggested effect size measures (Cohen's ω only when based on two ability groups) yielded acceptable to conservative type-I-error rates, whereas, the canonical correlation outperformed Cohen's ω in terms of empirical power. The simulation results further suggest that an effect size threshold of 0.30 is more appropriate as compared to more lenient (0.10) or stricter thresholds (0.50). The suggested item-analysis procedure is illustrated with an analysis of twelve Raven's progressive matrices items in a sample of N = 499 participants. Finally, strategies for item selection for cognitive ability tests with the goal of scaling by means of nested logit models are discussed.